Slow or unresponsive TEM Clients
Sometimes, you might see the symptom where the TEM Clients are slow to take actions, such as when an patch remediation is issued.
The most common issues are the 2 below that are worth understanding and to investigate further.
TEM Server cannot notify the TEM clients of new actions
The TEM platform is technically a pull technology, meaning that the TEM Clients are always the one downloading instructions, and retrieving software payloads. However, when new actions are issued, the TEM Server will notify the TEM Clients via UDP. By default, the TEM Clients will go to the TEM Server to poll or “gather” new instructions once in 24 hours. This is why a TEM Client whom the TEM Server cannot reach, might still take action within 24 hours.
This is described as traffic pattern “B” in the TEM Network Traffic guide (https://www.ibm.com/developerworks/mydeveloperworks/wikis/home/wiki/Tivoli%20Endpoint%20Manager/page/Network%20Traffic%20Guide?lang=en)
So if the TEM Server/Relays are not able to send UDP messages to the TEM Clients due to network constraints the Gather Interval can be modified from the default of once in 24 hours.
This can be configured via the TEM Client settings so this can be done on a client basis:
Please see the following TEM Support Site content which can be used to enable this setting:
* ID 157: BES Client Setting: Enable Command Polling
To enable this setting manually please see the following setting:
_BESClient_Comm_CommandPollEnable
_BESClient_Comm_CommandPollIntervalSeconds
The CommandPollEnable of value "1" will enable this mechanism and a value of "0" will disable it. The CommandPollIntervalSeconds setting determines how often the client will check with its TEM Server/Relay for any new gathers or refreshes. It is specified in seconds, such as 14400 for every 4 hours (14,400 seconds)
Please refer to the following document for how to create a computer setting:
http://support.bigfix.com/bes/misc/besconfigsettings.html
Important: You should not set the Gather Interval less than 10 minutes otherwise the TEM Clients may not function properly because they will constantly interrupt themselves to gather the latest actions.
Note: You can also change the Gather Interval value in the masthead using TEM Admin so that all TEM Clients (including any TEM Clients that will be installed in the future) will use the specified Gather Interval. Please note that the masthead changes the gather interval for the TEM Client's Actionsite (commands issued by the Master Operators). It does not change the gathering interval for the Opsites or Fixlet sites. To change the gather interval for all subscribed sites, use the _BESClient_Comm_CommandPollEnable setting described above.
Another reason to use these settings would be if the TEM clients are in the DMZ and cannot receive UDP messages from the TEM relay.
(Reference: http://support.bigfix.com/cgi-bin/kbdirect.pl?id=185)
TEM Clients are too busy evaluating content
The TEM Client is in a constant loop of evaluating each Fixlet message, Task, Baseline, etc. to see if it is relevant. Sometimes, due to excessive number of Fixlets subscribed, or inadvertently written Fixlets that spends an inordinate amount of time, the TEM Clients can appear unresponsive. For example, if a Fixlet is written to scan the entire file system or frequently querying network resources that might not be available.
Determine the Client Evaluation Cycle
There is a new Relevance Inspector in TEM 8.0 that can provide useful information regarding the Client Evaluation Cycle. An Evaluation Cycle represents a complete run through all the content available on the TEM Client, measured in milliseconds. The Inspector returns statistics based on the time sampled whenever the client returns to the beginning of its content set.
The Relevance “average of evaluationcycle of client” will return the average evaluation cycle for the last 10 loops. Anything in the 15 minute range is good. Anything over 30 minutes is worthy of investigation. Use the Client Profiler to debug Fixlets taking too long to evaluate.
Client Usage Profiler
The client usage profiler is an important tool (enabled via setting) that will log time spent on evaluating content. This information allows you to see which Fixlet messages, tasks, actions, or properties consume most of the Client's time. After the usage profiler has been enabled, the Client tracks the top 100 activities that take the longest time and stores them in a file called "usageprofiler.txt.xxxx", located in the same folder as the Client.
Note: After enabling the usage profiler, you may have to restart the Client before it is able to produce the log file "usageprofiler.txt.xxxx" and begin tracking activities.
Logging can be enabled and disabled using the following Tasks from the TEM Support Site in your Console:
ID 361: TROUBLESHOOTING: Enable BES Client Usage Profiler
ID 418: TROUBLESHOOTING: Disable BES Client Usage Profiler
Please see the following example of a Usage file that contains unusually long evaluation times. A health item would take less than a second to evaluate. Items that take longer should be investigated further.
Start:Thu, 30 Feb 2011 15:21:40 -0800
Elapsed Time:02:48:48
Tracking: Top 100
Samples:99
Elapsed Evaluation Time:18:18:32
1) Time in Milliseconds:Site.Item ID:Activity
2) 16478.126: actionsite.2147453787:Analysis Relevance
3) 16478.124: actionsite.2146599782:Analysis Relevance
4) 16478.119: actionsite.2142786964:Analysis Relevance
5) 16478.118: BES Support.521:Verify Fixlet Relevance
(Reference: http://support.bigfix.com/cgi-bin/kbdirect.pl?id=1802)
Client Debug Logging
For advanced troubleshooting purposes, it is sometimes useful to enable the debug log on a TEM Client so it will log every activity.
To turn on debug logging through the TEM Console, follow the instructions in the Task in the TEM Console titled:
* ID 157: BES Client Setting: Enable Debug Logging
* ID 196: BES Client Setting: Disable Debug Logging
To turn on Message debug logging manually, go to the computers tab in the TEM Console and create a TEM Client custom setting (choose the computers, right-click and go to "Edit Computer Settings"). Create two custom settings wiith the names "_BESClient_EMsg_Detail" and "_BESClient_EMsg_File" and the values as specified below:
_BESClient_EMsg_Detail - This configuration setting will enable the TEM Client debug log that will give extended information about TEM Client activity. The higher the level of detail, the greater the detail of the messages that are put into the log. The value should be set between 0 and 10000. Use 10,000 by default.
_BESClient_EMsg_File - Full path to file to store activity messages. The _BESClient_EMsg_Detail setting must be greater than 0 to use this option. The value should be the full path to the log (Example: C:\BESClientEMsg.log)
Note: If the TEM Client does not appear in the TEM Console, you can still turn on the Message debug Log by manually creating the client settings in the registry:
[HKEY_LOCAL_MACHINE\SOFTWARE\BigFix\EnterpriseClient\Settings\Client\_BESClient_EMsg_Detail] "value"="10000"
[HKEY_LOCAL_MACHINE\SOFTWARE\BigFix\EnterpriseClient\Settings\Client\_BESClient_EMsg_File] "value"="c:\BESClientDebug.log"
(Reference: http://support.bigfix.com/cgi-bin/kbdirect.pl?id=186)